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In parent-child communication, emotions are evoked by various types of intrinsic and 
extrinsic motivation. Those emotions encourage actions that promote more interactions. 
We present a motivation model of infant-caregiver interactions, in which relatedness, 
one of the most important basic psychological needs, is a variable that increases 
with experiences of emotion sharing. Besides being an important factor of pleasure, 
relatedness is a meta-factor that affects other factors such as stress and emotional 
mirroring. The proposed model is implemented in an artificial agent equipped with a 
system to recognize gestures and facial expressions. The baby-like agent successfully 
interacts with an actual human and adversely reacts when the caregiver suddenly 
ceases facial expressions, similar to the "still-face paradigm" demonstrated by infants in 
psychological experiments. In the simulation experiment, two agents, each controlled by 
the proposed motivation model, show relatedness-dependent emotional communication 
that mimics actual human communication. 
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1. INTRODUCTION 

Humans acquire knowledge and skills voluntarily by interacting 
with the environment. This voluntary learning process is driven 
by intrinsic motivation, which embodies curiosity and interest. 
By contrast, extrinsic motivation results in rewards such as food. 
White (1959) proposed that the intrinsic desire to interact with 
the environment and others underlies human exploratory behav- 
ior. Intrinsic motivation encourages individuals to seek novelty, 
uncertainty, and complexity (Berlyne, 1960). According to the 
self-determination theory of Ryan and Deci (2000), humans have 
three inherent fundamental needs: autonomy, competence, and 
relatedness. Autonomy is the perception that one's behavior is 
compatible with one's approval. Competence is fulfilled when 
expected or desired results are achieved. Relatedness is gained 
when one senses a close relationship with others. Ryan and Deci 
insist that these fundamental needs and individual differences are 
shaped by the social context. 

Fundamental needs are closely related to emotions. Reis et al. 
(2000) showed that satisfaction levels of fundamental needs are 
correlated with emotional evaluation indices. Interestingly, while 
the satisfaction levels of autonomy and competence correlate 
with both positive and negative emotions, the relatedness level 
correlates only with positive emotions. Closely related persons 
evoke more emotions than strangers. If relatedness is not satis- 
fied, unpleasant emotions are not necessarily evoked, but people 
sense discomfort when an expected reaction is not delivered by 
the related person. Thus, compared with the other two needs, 
relatedness exerts a more complicated effect on emotions. 

The need for relatedness becomes apparent from the early 
stage of infant development. Still-face paradigm experiments have 
shown that infants are socially sensitive to others (Adamson and 
Frick, 2003; Striano, 2004). In these experiments, the caregiver 



suddenly ceases normal interaction with the infant and shows a 
still face. Throughout this phase, the caregiver reduces the num- 
ber and extent of positive activities, such as smiles or attention. 
Infants react to this behavior with restore reactions such as clap- 
ping or reaching to the caregiver to draw their attention. The 
reactions shown by infants depend on their development stages 
(Adamson and Frick, 2003). These experiments show that infants 
are motivated to establish relatedness with others and that they 
require attachment to others. 

Several studies in cognitive developmental robotics (Asada 
et al, 2009) have sought to understand initial communication 
by computational models (Ogino et al., 2007; Watanabe et al., 
2007). However, these studies focused on acquiring communica- 
tive actions, rather than the factors that motivate communication. 
The mechanism that encourages an agent to behave according to 
internal discipline rather than external reward has been identified 
as intrinsic motivation (Barto et al., 2004; Oudeyer et al., 2007). 
However, intrinsic motivation studies continue to adopt self- 
learning tasks such as skill acquisition. The question remains: how 
do intrinsic motivation mechanisms promote communicative 
interactions? 

This paper proposes a motivation model of early communica- 
tion between an infant and his/her parent, in which the need for 
relatedness triggers emotional change and behavior learning. The 
proposed model aims for dynamic interaction between two agents 
who estimate each other's the internal state. Throughout the 
interaction, an interpersonal relationship is established in which 
approaching and sharing another's emotion encourages interest 
and relatedness to him/her, alters emotional states, and pro- 
motes mutual behavior. While relatedness directly and indirectly 
affects the emotional state of an agent, it also changes the reward 
for action selection. We consider that the network of dopamine 



www.frontiersin.org September 2013 | Volume 4 | Article 618 | 1 



Ogino et al. 



Relatedness-based motivation model for interaction 



Parent 




Child 



Face Expression 

/ 



Internal Rev\ 



Action (X 




Observations 
X 



Internal Variables 



Internal Rewards 

R a R a 



Novelty 


nov 


Relatedness 


rel 


Emotion Prediction 


^other 



X 



nov 
rel 

Mother 



Interaction scene 



Other's Emotion 

T" 



Other's Action 

-7 



Emotion Elements 



Novelty 



Expectation 



Facial 
Expression 



Overview of emotion system 



Re 



FIGURE 1 I (A) Variables used in the proposed model in an interaction 
situation. (B) Overview of the emotion system. 



neurons plays an important role in activating communication. 
Dopamine neurons are known to code the prediction error for 
reward in reinforcement learning (Schultz, 1998). In robotics, 
(Kaplan and Oudeyer, 2007) hypothesized that dopamine neu- 
rons encode signals for encouraging behavior that decreases the 
prediction error. Recent studies reveal that dopamine neurons 
are activated not only by explicit reward but also by novel sig- 
nals that are not directly related to these rewards (Dayan and 
Balleine, 2002; Kakade and Dayan, 2002). This indicates that 
dopamine neurons play an important role in intrinsic motivation. 
Dopamine neurons are also associated with emotional reactions 
in the amygdala (Phillips et al, 2010). From these neuroscience 
findings, it is reasonable to consider a model in which a variable 
corresponding to dopamine neurons mediates emotional change 
and behavior. In parentaA§infant interactions, the activation of 
dopamine neurons will arouse the infant's interest, and the parent 
will act to maintain this interest. 

2. MATERIALS AND METHODS 

2.1. MOTIVATION MODEL OF PARENT-INFANT INTERACTION 

In the communication situation of this study, an infant atten- 
tively interacts with his/her caregiver and displays emotional 
facial expressions such as laughing and crying. The interaction 
situation and variables used in the proposed model are shown 
in Figure 1A. The infant and the parent update their internal 
state, e, based on the observed information, x, and output their 
facial expressions, /, and actions, a. The facial expressions and 
actions are assumed to be produced and observed independently. 
The facial expressions are based on the agent's internal state, e, 
which partly depends on the facial expressions of the other agent, 
Mother- We suppose that both agents (parent and child) possess the 
same emotional system, comprising emotional elements, emotion, 
and action selection modules (Figure IB). The emotional elements 
module contains two main elements for intrinsic needs, Novelty 
and Relatedness, and other three sub-elements, Stress, Emotion 
Mirror and Expectation. The value of each element is determined 
by the other's facial expressions and actions. The emotion ele- 
ments are used to compute the current emotional state of the 
agent in the emotion module. Finally, in the action module, the 
reward value is evaluated from the emotional elements (pleasure 
and arousal), and gesture and facial expressions are selected. The 
following subsections describe the mechanisms of the internal 
state. 

2.1.1. Emotion 

Russell (1980) proposed that all emotional states lie within a 
two-dimensional space comprising an arousal-sleep axis and a 
pleasure-unpleasure axis. Following Russell's model, we define 
the emotional state e as a vector of arousal and pleasure elements. 




The emotional state is updated by the reward, R e , as follows; 

e(t+l) = e(t) + i\(Re(t)-e(f)). (2) 



The elements of the reward corresponding to arousal and plea- 
sure, R% • R p , are composed of various psychological factors — 
novelty, relatedness, emotional contagion, and expectancy — 
denoted nov, rel, e const , str, and £ gra d> respectively. 

Rj(t) = a A nov(0 + p A str(f) + y<4 nt (f) (3) 
R p e (t) = a p rel(0 + P% ad (0 + ye P cont (t). (4) 

The novelty, nov, indicates the degree of interest in novel sur- 
rounding objects, and it is defined as 

nov(f) = 1/(1 + exp(-m(/(f) - 6)), (5) 

where I(t) is information gain, and m and 0 are constants. 
The information gain, I(t), is based on a state transition model 
constructed by the agent's observations, 

J(f) = -logp(s(f+l) I 5(0). (6) 

Stern (1985) proposed that the emotional attunement of a par- 
ent is important in establishing a parent-child relationship. Such 
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emotional attunement is thought to be necessary for the sharing 
mind states. Thus, we assume that the relatedness variable, rel, 
depends on the synchronization of emotional states: 



rel(t) = (1 - |x)rel(f - 1) + vsim(f), 



(7) 



where |x and v are constants, sim is the emotional similarity, i.e., 
the extent to which other's emotional states are shared between 
the agents, sim is the inner product of the self and other's emotion 
vectors: 



sim(f) = e(f)e ot her(0. 



(8) 



As suggested by fMRI experiments (Singer et al., 2004), humans 
possess an emotional mirror system. A person's emotional state 
is slightly altered by the perceived emotional state of another. 
In this paper, the variable for emotional contagion variable, 
^cont) is the product of relatedness and the emotional state of 
the other: 



e C ont(0 = rel(0 Mother (0- 



(9) 



Note that the emotional contagion increases with increasing 
degree of relatedness. 

When a parent is unwilling to relate to his/her infant, the 
heart rate of the infant increases, and the infant's gaze is averted 
from the parent, apparently because the infant is temporarily 
aroused by the stress of communication failure (Field, 1981). In 
our model, the stress variable increases when emotional sharing 
with the related person fails; that is 



str(f) = rel(f) exp(— asim(f)). 



(10) 



where a is a positive constant. 

While emotional contagion and stress cause temporary effects, 
the impact of emotional expectancy is long lasting. For exam- 
ple, pleasure is enhanced when one's action appears to please 
another. Thus, we define emotional expectancy as temporal gra- 
dient of expected pleasure, defined by multiplying the action 
selection probability by the pleasure of the other at the present 
and preceding moments, and taking their difference. 

£grad(0 = rel(f) (pflW^therW " Pa(f ~ ^other^ " *b)) ( U ) 

The emotional expectancy is large when the action selection prob- 
ability and the pleasure emotion increase together. Emotional 
expectancy is also affected by relatedness. 

2. 1.2. Motivation mechanism for action 

Dopamine neurons in the midbrain are considered to encode 
values; they are activated and suppressed in desirable and 
undesirable situations, respectively. However, some dopamine 
neurons have recently been reported as activated even in unde- 
sirable situations. Bromberg-Martin et al. (2010) proposed that 
dopamine neurons encode either motivational value or motiva- 
tional salience. Thus, we model two classes of dopamine neurons, 
as follows. 



Dopamine neurons belonging to the first class, encoding a 
motivational value, are projected from the basal ganglia and 
contribute to the exploratory and evaluative learning of whether 
the current situation is desirable/undesirable. In infant-parent 
communication, actions that attract the infant's interest and 
establish relatedness will score high motivational value. Thus, 
we suppose that the first class of dopamine neurons encodes the 
other's arousal emotion and relatedness, 



< alue (0 = ^ ther (0 + ^rel(0, 



(12) 



where co is a positive constant. 

Dopamine neurons belonging to the second class, motiva- 
tional salience, are projected from the amygdala. The neurons 
contribute to the learning of motivationally important events that 
may not be related to reward and are thought to aid attention and 
working memory. We suppose that the second class of dopamine 
neurons encodes the arousal emotion 



^Salience (f) = (A (t y 

Both rewards are summed to give the total reward 



Rait) 



pR v /» e (t) + (1 



. p) £Salience (0? 



(13) 



(14) 



where p is a weighting constant (0 < p < 1). As p increases, an 
agent acts upon predictions of the other's emotional state. If p is 
small, an agent acts more upon its own emotional response. 

Reinforcement learning is used to update the action policy. 
Although various sensor information is important in actual com- 
munication, here we consider actions alone. When an action a 
yields a reward R a , the corresponding action value function R a is 
updated as 

R a (t + 1) <- Ra(t) + r\ Ra (R a (t) - Ra(tj) , (15) 

where r\R a is a learning coefficient. 

The parent agent adopts an e -greedy policy. That is, with prob- 
ability s, the parent agent selects the highest- valued action, R a , 
among its own action repertoire, and otherwise chooses random 
actions, 



7C parent (0 



{random action (t, < e) 
argmax a R a (t) (otherwise) 



(16) 



where t, is randomly drawn from a uniform distribution in the 
interval [0, 1]. 

The actions performed by the infant agent depend on the 
action value R a . The movements of infants appear random and 
occasional, whereas those of their parents are voluntary. Thus, 
we model the selection and performance of infant actions by a 
Boltzmann equation 



child 0) 



exp 



(X(0/t) 



c + exp 



(i(0/t)' 



(17) 
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where c is the initial probability that an action is taken. The 
temperature parameter x determines the randomness of action 
selection. 

2.2. INTERACTION EXPERIMENT WITH A VIRTUAL ROBOT 

To validate its applicability in real communication, the proposed 
model was implemented in a virtual agent. The virtual agent com- 
municates with a human experimenter who mimics parent-like 
facial expressions and behavior. 

The experimental setup is shown in Figure 2. Displayed on 
a laptop computer, the virtual agent (Figure 2A) observes facial 
expressions and behavior of the experimenter by a USB camera 
attached to the top of the laptop display. 

The virtual agent displays four types of facial expressions 
depending on its emotional state. It also exhibits an appealing 
behavior by arm movement. The facial expressions and appealing 
behavior are shown in Figure 3. 

The experiment was undertaken in two phases. In the first 
(learning) phase, the virtual agent learns the relationship between 
the experimenter's facial expressions and its corresponding emo- 
tional states and constructs a layered network for behavior 
recognition. In the second (interaction) phase, the virtual agent 
communicates with the experimenter. 

In the learning phase, information for emotional estimation 
and behavior detection is processed from camera images. During 



emotional estimation, the estimated emotional state of the exper- 
imenter, e ot her> is output from the camera image, x. The facial area 
in the captured image is extracted by the facial recognition algo- 
rithm in OpenCV (Bradski, 2000), converted to a gray scale image 
of size 128 x 128 pixels, and binarized by a specified threshold. 
In the learning phase, a certain number of facial images, i z , is 
recorded and each is stored with its corresponding emotional 
state, e{. The correspondence between the emotions of the virtual 
and human agents is learned by imitation (Watanabe et al., 2007); 
that is, the human agent imitates the facial expressions of the vir- 
tual agent when presented with a stimulus such as a blue object or 
keyboard pressing (these responses of the virtual agents are pre- 
programmed). In the interaction phase, the input facial image I x 
is compared with the stored images and the best-matched facial 
image is selected as 

I min = argmin \I X - J;| 2 (18) 

Let \|/ be the mapping function. The momentary estimated emo- 
tional state of the experimenter is calculated as 

enow = \Kfmin). ( 19 ) 

The estimated emotional state is the temporal average of the 
momentary estimated emotional states, 




Mother = (1 — S) e ot her + &0 n 



(20) 



FIGURE 2 | Experimental setup. (A) Virtual agent. (B) Communication 
scene. 




FIGURE 3 | Facial expressions and appealing behavior of the virtual 
agent. (A) Calm, (B) laugh, (C) surprise, (D) cry, and (E) appealing. 



where 8 is an update constant. 

Figure 4 shows the learned map of the facial images and their 
corresponding emotional states. The vertical and horizontal axes 
indicate the arousal and pleasure levels, respectively. For this 
experiment, the emotional state of the experimenter is estimated 
from 25 images. 




FIGURE 4 | Learned map of facial images and emotional states. 
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Behavior recognition is achieved by a layered neural net- 
work of slow feature analysis (SFA). The SFA learning algorithm 
extracts the slowly changing components from the input signals 
and estimates the inherent information based on their statisti- 
cal properties (Wiskott and Sejnowski, 2002). According to some 
studies, SFA exhibits stronger gesture recognition performance 
than existing methods such as the hidden Markov model and 
random forest (Koch et al, 2010). 

The input to the SFA layered network is an image of the 
experimenter waving a red object in his hand in various direc- 
tions. The learning data are 2000 steps of images. The input 
image (320 x 240 pixels) is segmented into the small areas of 
size 16 x 12 pixels. Each small area is labeled as "1" if the num- 
ber of red pixels (specified by RGB content R > 160G < 50B 
< 80) exceeds half; otherwise, it is labeled "0". The resultant 
128 -dimensional vector is used to construct a state transition 
model. The range of the j-th unit in the SFA output layer, yj, is 
divided into Sj bins. The output signal is described by the discrete 
states s(yj) (s e {1, 2, . . . , Sj}). from which the state transition 
probability in the j-th output signal, yj, is calculated as 

p/ ss , = Pr{s ( yj (t + 1)) = sf | s ( yj (t)) = s}. (21) 

This state transition model is iteratively updated when a new state 
is observed. 

The information gain of yj, Ij(t), is calculated by the state 
transition model as 

mt) = - 1 ^Y! t=t - ta ^p{s(y j {t+\)) \s(y j {t))). (22) 

From 22, the novelty of the j-th output signal is evaluated as 

nov A f > = i + exp(-m(j ;(t )-e)) - ( M ) 

Finally, the novelty of the whole output signal, nov(f), is calcu- 
lated as the average of the novelty of each output signal 



nov(0 = ±E"nov,(0 



(24) 



in developmental psychology, passing through the three phases of 
interaction, still face, and reunion. During the interaction phase, 
the experimenter looks at the camera and expresses surprise, sim- 
ulating a parent seeking the attention of his/her infant. Then, 
when the agent similarly expresses surprise, the experimenter 
ensures that the arousal emotion is shared and begins laughing to 
the virtual agent. Throughout the interaction, the experimenter 
moves the red object, starting with the action patterns shown in 
Figure 5, and later by free motion. The persistent changes in the 
action pattern maintain the arousal level and the attention of the 
virtual infant. 

During the second phase (still face), the experimenter ceases 
object movement and shows a blank facial expression. The 
possible unfamiliarity between experimenter and agent is non- 
problematic, because in actual still-face paradigm experiments, 
the still- face effect is elicited in infants meeting a person for the 
first time (Adamson and Frick, 2003). 

During the last phase (reunion), the experimenter reverts to 
the interaction phase; that is, moving the red object and present- 
ing emotional facial expressions. 

The virtual agent shows simulates laughing (eP > 0.4), cry- 
ing (e 9 < 0), surprise (0 < £ < 0.4, > 0.4), and normal 
(otherwise). The relationship between the emotional states and 
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During the interaction phase, the experimenter communicates 
with the virtual infant agent with a red object in his/her hand. 
The communication mimics the still- face paradigm experiment 



FIGURE 6 | Facial expressions and emotional states of a virtual infant 
agent. 




Action <i2 



FIGURE 5 | Actions made by experimenter. (A) Action ao> (B) Action ai . (C) Action a 2 . 
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the facial expressions is shown in Figure 6. The agent appeals 
(Figure 3E) to the experimenter based on the probability of 
action taken (Equation 17). 

The simulated still- face paradigm experiment was conducted 
over the time frame of the equivalent developmental psychol- 
ogy experiment (Adamson and Frick, 2003); 2 min for the first 
interaction, 2 min for the still face, and 2 min for the reunion. 

While the emotional expressions and actions of the human 
experimenter are continuous, they are undertaken by the virtual 
agent in a numerical time step (430 ms). To maintain natu- 
ral interaction, the agent retains the triggered facial expression 
and action for 4 s. The experimenter's emotional state is esti- 
mated every 5 steps. The emotional state of the agent and all 
other variables are updated at each step. The novelty is eval- 
uated after each 30-step sequence of human actions (i.e., the 
t a is set to 30 in Equation 22). The constants and param- 
eter settings of the infant agent are described in the next 
section. 

2.3. SIMULATION EXPERIMENT OF PARENT-INFANT INTERACTION 

We suppose that parent-infant interaction is enabled by a moti- 
vation mechanism that is common to both individuals. The com- 
munication dynamics are governed by mutual attraction between 
the infant's and parent's motivation. In this section, the pro- 
posed model is implemented in two agents to determine whether 
the parent-infant interaction emerges through interplay between 
emotion and action in a simulation environment. We also exam- 
ine how the relatedness of the infant agent changes in response to 
varying patterns of action and emotional expressions presented 
by the parent agent. 

Both agents are assigned three actions, ao, a\ and A2> as shown 
in Figure 5. The parent agent selects its action from the reper- 
toire when the action value is updated. If the probability of action 
(Equationl7) exceeds a given threshold, the infant agent adopts 
the action taken by the parent in the previous step. 

Action recognition is based on the image sequence recorded 
in the interaction experiment between the human and the vir- 
tual agent. When its partner performs an action, the observing 
agent accepts an image sequence (30 images 9 of the active agent 
as input. When no action of the agent is observed, the novelty of 
the observer decreased by a factor of X, 

nov(f) = A,nov(f — 1) (if no action is observed). (25) 

In emotional expression and recognition, we assume for simplic- 
ity that one agent can observe the emotion of another agent, e 0 ther> 
from his/her facial expression, pother without mistakes. 

We also assume that two steps of simulation time correspond 
to 1 s. An action is selected, and the action value, together with 
the emotional estimate of another agent, is updated every 5 steps. 
All other variables, including the emotional state, are updated at 
each step. 

We allocated the following five conditions of facial expression 
and action patterns of a parent agent. 

1. normal 

2. still face 



3. fixed action 

4. random emotion/action 

5. no relatedness 

Under condition ( 1 ) normal, the parent agent behaves according 
to the proposed emotional system. 

Under condition (2) still face, the parent agent adopts the 
still-face behavior in human-agent interaction experiments. The 
simulated experiment is undertaken in three phases; interac- 
tion phase (0-999 steps), still face phase (1000-1199 steps), and 
reunion phase (1200-1399 steps). Each phase corresponds to 2-3 
min in real time. While both agents follow the proposed model 
during the interaction phase, the parent ceases facial expression 
and activity in the still-face phase. During this phase, the emo- 
tional state of the parent agent is set to = 0 and eP = 0. In the 
reunion phase, the parent agent recovers its emotional expression 
and resumes action. 

Under condition (3), fixed action, the parent agent selects the 
same action, a\, while its emotional expressions are governed 
by the proposed model. Unlike the normal condition, in which 
action selection by the parent depends on the action value, R a , the 
fixed action arouses marginal emotion in the infant. The resulting 
lack of novelty perceived by the infant reduces the relatedness. 

Under condition (4), random emotion/ action, the parent 
expresses random emotion expressions and performs actions ran- 
domly. The arousal and pleasure values are randomly selected 
from —1 to 1. Among the three-action repertoire, each action is 
selected with equal probability. While emotions are continuously 
shared between the parent and infant agents under normal con- 
ditions, emotional sharing is interrupted under this condition. 

Under condition (5), no relation, the relatedness of the par- 
ent agent is not updated (and remains fixed at 0). This condition 
enables the observation of how relatedness between the agents 
affects their emotional sharing. 

The parameters and coefficients used in this experiment are 
listed in Tables 1-4. 

3. RESULTS 

3.1. EXPERIMENTAL RESULTS IN INTERACTION EXPERIMENT WITH 
VIRTUAL ROBOT 

Throughout the 6 -min interaction period, the virtual agent com- 
pleted 828 calculation steps. Figure 7 shows the temporal profiles 
of relatedness during the interaction. Throughout the first inter- 
action phase, the relatedness increases to its maximum value 
1.0 in 118 s. The relatedness declines throughout the still-face 
phase (from 120 to 240 s) is minimized (0.33) at 247 s and 



Table 1 | Parameters of emotional elements. 


Parameter 


Explanation 


Parent/ 






Infant 


m 


Coefficient of information gain for novelty 


100 


0 


Threshold of information gain for novelty 


0.9 


[I 


Decay constant for relatedness 


0.006 


V 


Coefficient of vector similarity for relatedness 


0.025 


G 


Coefficient of similarity for stress 


5 
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Table 2 | Parameters of emotional change. 



Parameter 


Explanation 


Parent 


Infant 


a A 


Coefficient of novelty for arousal 


0.8 


0.5 




reward 








Coefficient of stress for arousal reward 


0 


2 


a p 


Coefficient of relatedness for pleasure 


0.8 


0.45 




rewa rd 






P P 


Coefficient of expectancy for pleasure 


0 


40 




reward 






Y 


Coefficient of emotional contagion for 


0.6 


1 




pleasure reward 








Coefficient for update of emotional 


0.03 




state 






Table 3 | Parameters of action motivation. 


Parameter 


Explanation 


Parent 


Infant 


CO 


Coefficient of relatedness for 


0.3 






motivational value 






p 


Weight of motivational value for action 


1 


0 




reward 








Coefficient of action value update 


0.6 


s 


Probability that parent selects random 


0.1 






action 






c 


Initial constant of action selection of 




4 




infant 






X 


Temperature constant of action 




0.3 




occurrence probability of infant 






Table 4 | Other system parameters. 


Parameter 


Explanation 




Parent/ 








Infant 


n 


Number of input signals for novelty detection 


20 


Sj 


Number of bins in input signals for novelty detection 


20 


Tie 


Coefficient for emotion estimation 




0.05 



recovers throughout the reunion phase (after 240 s) when normal 
interaction is resumed. 

Figure 8A shows the emotional state of the experimenter esti- 
mated by the infant agent. While the experimenter shows a 
positive emotional state in the interaction and reunion phases, its 
arousal and pleasure value fall to 0 during the still-face phase. 

Figure 8B shows how the emotional state of the infant vir- 
tual agent changes over time. During the first interaction phase, 
positive emotion continues, and the pleasure level increases with 
increasing relatedness. Note that the arousal level suddenly esca- 
lates in the still-face phase, while the pleasure level decreases. 
During the reunion phase, the arousal settles around 0.5, and the 
pleasure recovers. Figure 9 shows the probability of action taken 
by the agent. This probability increases with increasing pleasure 
level throughout the interaction phase, but suddenly leaps in 
the still-face phase. This trend mirrors the appealing behavior of 
infants real-time still-face experiments. 
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FIGURE 7 | Relatedness of virtual agent as a function of time in the 
simulated still-face experiment. 



3.2. EXPERIMENTAL RESULTS OF SIMULATED PARENT-INFANT 
INTERACTION 

Figure 10 shows the temporal dynamics of relatedness in the 
infant agent while interacting with the parent agent under the 
five conditions. While the relatedness increases to its maximum as 
the interaction proceeds under normal and still-face conditions, 
it remains low under the remaining three conditions. During 
the 860 steps of the interaction phase under the sitll-face con- 
dition (corresponding to the normal condition), the relatedness 
increases to 1 . However, while the relatedness remains at 1 under 
normal conditions, it declines throughout the still- face phase, 
because the parent shows not emotional expression, and the 
degree of emotional sharing, sim, reduces to 0. In the reunion 
phase, after 1202 steps, the relatedness recovers as observed in the 
human-robot interaction experiment. 

Figure 1 1 shows the emotional states of infant and parent 
agents. Throughout the interaction phase, the actions of the par- 
ent engage the infant agent, raising its arousal level. The increased 
relatedness enhances the pleasure level in both agents. During the 
still- face phase, both the arousal and pleasure levels of the parent 
agent decrease to 0 (Figure 1 1A). On the other hand, the resulting 
stress to the infant (described by Equation 10) increase its arousal 
level (Figure 11B). Increased arousal is accompanied by a decline 
in the pleasure level shortly after entering the still-face phase. This 
negative emotion is induced by the negative value of expectancy 
value (Equation 11). During the reunion phase, the arousal level 
of the infant decreases to pre-stress levels, and the pleasure level is 
recovered as relatedness is restored. 

Under the fixed action condition, the relatedness increases to 
0.1 and gradually declines to a low level. By contrast, related- 
ness remains low under random action/ emotion conditions. As 
defined in Equation (7), relatedness is determined by the sim- 
ilarity of emotional states between the two agents. Throughout 
the interaction phase, the arousal level of both agents necessar- 
ily increases, as specified in the still- face condition. However, 
when the parent performs fixed actions, it stimulates no novelty 
in the infant. Although random actions do stimulate novelty in 
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simulated still-face experiment. 



the infant, the randomness of the parent's emotional expressions 
interrupts emotional sharing, thereby reducing the relatedness 
under random action/ emotion conditions. 

Under the no relatedness condition, the relatedness of the 
infant agent increases up to around 0.2 during the first 400 steps 
and remains at 0.2 thereafter. During the interaction phase, the 
shared arousal emotion enhances the relatedness. Subsequently, 
the shared pleasure emotion further increases the similarity, sim, 
and thus the relatedness. However, since relatedness dominates 
the pleasure level, pleasure cannot increase if the parent lacks 
relatedness. Thus, high relatedness in the infant agent can be 
achieved only by the sharing of arousal emotion. 

4. DISCUSSION AND CONCLUSION 

In our model, parent infant interactions are primarily mediated 
through novelty and relatedness. Novelty motivates interaction 
with the environment. Since the novelty value is evaluated from 
a pre-learned state transition model, it is increased by the per- 
ception of dynamic movement and reduced in still environments. 
Based on this property, the parent predicts which action will 
elicit higher novelty in an infant, such as moving an object. 



As the infant detects novelty in his/her parent's behavior, its 
arousal level and response frequency are enhanced. In turn, the 
infant responses evoke novelty, and hence arousal, in the parent. 
Increased arousal in both agents increases emotional sharing, and 
hence the relatedness, between the agents. This enhanced relat- 
edness encourages pleasurable emotions and further emotional 
sharing. The simulation experiment demonstrated this positive 
feedback effect of mutually exchanged rewards. 

In the proposed model, novelty and the state transition prob- 
abilities of other agent's actions are evaluated by SFA networks. 
Such networks are effective for extracting similar action structures 
from image sequences, because they can integrate temporarily 
similar information. This property of SFA networks renders them 
suitable for gesture recognition, where repeat observations of the 
same action are perturbed by human motion and lighting con- 
ditions. In fact, unvarying repeated action decreases the novelty, 
because the same state transition is observed. 

The relatedness modeled in this paper does not consider 
long-term relationships. We reiterate that the still-face paradigm 
is applicable not only to parent-infant interactions but also 
to stranger-infant interactions (Adamson and Frick, 2003). 
Furthermore, the still- face response is absent during interactions 
with impersonal objects. This finding indicates that infants can 
relatively quickly identify whether an object/person is amenable 
to social interaction and can related to that object or per- 
son. Humans do not empathize with objects and other humans 
that fail to comply with expectation, unless relatedness is also 
present. Relatedness is regarded as a precursor to all social emo- 
tions, including social expectation, social contagion, and social 
stress. For this reason, the modeled terms of social contagion, 
stress, and expectation of emotional reward include multiples of 
relatedness. 

An interesting result of the proposed model is that surprise 
appears first in the interaction, followed by pleasure. This is 
attributable to the evocation of arousal by novelty detection, 
which occurs regardless of relatedness, while the pleasure emo- 
tion arises only through relatedness. Thus, during in the initial 
interaction, when relatedness is low, arousal is elicited first. Next, 
as arousal is shared, the relatedness is increased, followed by plea- 
sure, which elicits the smiling response. In this way, emotional 
contagion encourages further emotional sharing. 
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FIGURE 10 | Relatedness during simulated parent-infant interactions. (A) Relatedness of infant agent. (B) Relatedness of parent agent. 
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FIGURE 11 | Emotional states during simulated still-face interactions. Red and green lines indicate the arousal and pleasure levels, respectively. (A) Parent 
agent. (B) Infant agent. 



Although the parent frequently changes action during inter- 
action phase, the frequency of change decreases as relatedness 
increases. High relatedness maintains the motivation at a high 
level and prevents the decline of the action value. Indeed, in the 
simulation experiment, the parent altered its actions 41 times 
through the interaction phase, increasing to 96 times when relat- 
edness was set to 0. In actual human communications, this trend 
might signify a shift from a unidirectional form, in which a par- 
ent attracts the attention of an infant, to a bidirectional form, in 
which both parent and child pursue pleasurable emotions. 

The simulation experiment investigated how the interaction 
between the parent-infant interaction changes when the related- 
ness of the parent agent is not updated. Under this condition, 
the emotional state of the parent is static, and the action pat- 
terns depend on emotional sharing with the infant. During the 
first phase, the arousal level increases in both parent and infant 
agents, increasing the relatedness and pleasure levels of the infant, 
while those of the parent remain fixed. In this case, because the 
emotional state vectors of both agents diverge, the relatedness 
and pleasure levels of the infant remain low. Thus, if one agent 
seeks relatedness and its accompanying pleasure, it must find 
another agent with the same goal at the same time. Baumeister 
and Leary (1995) proposed that human beings are fundamentally 
and pervasively motivated by a need to belong; that is, to form 
enduring interpersonal attachments. According to these authors, 
this need is satisfied when pleasant interactions occur within a 



temporally stable and enduring framework of affective concern 
for each other's welfare. In our simulation study, a similar recip- 
rocal relationship between two agents was required to maintain 
interpersonal attachments. 

In the simulation experiment, the relatedness was initialized to 
0 both in both agents. In an actual interaction, the parent who 
establishes communication with his/her infant possesses high 
relatedness at the beginning of the interaction. However, if the 
initial pleasure value of the parent agent is set to 1, the related- 
ness decreases, because the pleasure level does not match that 
of the infant. This occurs because relatedness in the proposed 
model depends only on emotional similarity. This problem might 
be solved by including a top-down mechanism, such as a bias 
term, when calculating the relatedness in the parent agent. Such 
a term would account for the parent's desire to interact with the 
infant. 

In this paper, the emotional state of an agent is defined in a 
two-dimensional plane whose aces are arousal and pleasure. This 
low- dimensional model of emotions has been previously adopted 
in robotics studies (Breazeal and Scassellati, 1999; Itoha et al., 
2005; Watanabe et al., 2007). In psychology, low- dimensional 
models are based on descriptive taxonomies and have proven rea- 
sonably successful for describing measures of self-reported emo- 
tion and relative confusion of various facial expressions. However, 
the sections of brain corresponds to each dimension are not 
clear. Arguably, such models cannot explain selective emotional 



w w w.f ro ntiersin.org 



September 2013 | Volume 4 | Article 618 | 9 



Ogino et al. 



Relatedness-based motivation model for interaction 



impairments (Calder et al., 2001). The difficulties in model- 
ing emotions necessitate a direct quantitative comparison of the 
model with psychological experiments. Facial expressions and 
physiological data such as Galvanic skin response are superfi- 
cial expressions of internal emotional states. In this paper, the 
still-face effect is qualitatively compared with the psychologi- 
cal still- face paradigm experiment. Although emotions appear 
to be dispersed within the human brain, unlike the phys- 
ical sense of touch, which is located in the somatosensory 
area, separated areas are probably connected within the state 
space of emotion. In future experiments, we plan to incor- 
porate brain mechanisms, including the relationships among 
brain regions related to emotion, and to compare the theoreti- 
cal model with brain activities during interactions (Dumas et al., 
2010). 

Gaze is one of the most important challenges in extending 
the proposed model. Arousal is closely related to attention. In 
the proposed model, an agent informs its interest to another by 
arousal- induced actions but does not inform the item of inter- 
est. Furthermore, the parent's action value varies over time, but it 
is independent of sensor information. Supplied with gaze infor- 
mation, a parent could locate and identify the item commanding 
the infant's attention, which would enrich communication. For 
example, parental behavior such as intentionally shifting the tim- 
ing of an action or showing exaggerated facial expressions after 



attracting the infant's attention would further enhance pleasure 
in the infant. 

The proposed model does not explain the decrease of the 
infant's attention toward the parent in the still-face phase. Such 
behavior is thought to decrease the stress experienced by the 
infant (Field, 1981). If true, our model must introduce attention 
mechanisms for controlling emotion. Furthermore, including 
gaze information, we could extend our simulated interactions 
from dyadic interactions to triadic relationships among par- 
ent, infant, and object. Especially, joint attention, in which the 
infant attends to the object occupying the parent's attention 
or promotes the parent to attend to his/her object of interest, 
is an important topic in the communication of shared emo- 
tion. The learning of joint attention has already been mod- 
eled in developmental cognitive robotics (Nagai et al, 2003; 
Triesch et al., 2006). In future extensions of our model, we 
aspire to understand how higher cognitive functions such as joint 
attention relate to motivational behavior such as novelty and 
relatedness. 
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